Direct vs indirect assessment of simple knowledge structures. ESD-TR-65-542.

This report compares two types of classroom testing in terms of efficacy in guiding instruction. One type of testing is the traditional indirect method based on the observation of choices. The other type is the direct method based on admissible probability measurement. The general finding is that the direct methods always perform as well as and in most cases better than the indirect methods. This deficiency in the indirect method can be alleviated in theory by introducing redundancy into the test and asking the same question over and over again. The performance of indirect methods depends in a very critical manner upon the information available to the instructor from other sources about the current state of knowledge of each student. The performance of the direct methods is unaffected by this. The gain in effectiveness achieved by using direct methods must be balanced off against the cost of using these new methods. A direct method may require more student time per item than does an indirect method. This, however, may be more than compensated for by the requirement for redundancy when using the indirect method. In addition, since a direct method does not require additional information from the instructor as to the current state of knowledge of each student, the possibility exists that much larger classes may be taught with no loss in effectiveness thus implying even further economic benefits from the use of direct methods to guide classroom instruction.